In general, the present invention relates to light projector systems used for information display on a surface. Most, if not all, light projector systems are subject to display distortion, whether caused by perturbations of an electronic and/or mechanical nature (e.g., vibrations common to most viewing environments, such as ventilation system operation, doors closing, and so on), resulting in display mis-calibration. Here, the continuous monitoring of an active display is done to acquire a model of the display surface while the display is in use. Light-projectors are being explored and used for more-demanding tasks in connection with augmented-reality applications, interactive displays, and groups of projectors employed to collectively render high-resolution immersive environments. The placement of projectors is often constrained by the amount of display distortion that can be tolerated. In order to avoid image distortion, a projector must either be placed so that the optic axis is perpendicular to the display surface plane or the display distortion must be corrected. Tilting a projector in the vertical plane induces a distortion commonly referred to as keystoning. Arbitrary placement of a projector yields an image that is governed by a more general projective distortion. Off-axis placement of light projectors induces significant planar parallax on the display surface. Although commodity solutions exist for removing this distortion, they involve iterative, menu-driven user interaction or physical alignment of the projector, and in either case interrupt the use of the display. User interaction is infeasible in a number of scenarios including mechanically aligned multi-projector displays that are subject to mechanical drift and perturbations; in those cases, projector(s) must be manually reconfigured, often.
More-particularly, the instant invention is directed to a technique and system for continuous rectification of arbitrary off-axis distortions that does not require user monitoring/interaction by incorporating at least one camera into the display system to function in a novel process to continuously monitor the state of calibration. Of interest, here, is to correct distortions due to the relative positioning of the projector with respect to the display surface. The (one or more) camera(s) automatically detects when the projector's orientation has changed, without requiring explicit fiducial or targets in the world coordinate system. The method runs in concert with interactive display applications and has minimal impact on frame-rate. An initial rectifying transform is recovered automatically by projecting target ‘points’, or fiducials/targets of a selected intensity/brightness distribution, and observing them within a camera. The display is then warped and passively monitored for calibration error and motion of the projector. The technique distinguishes between distortions due to miscalibration and intentional/desired projector framebuffer changes. A consistency/correlation/similarity score is calculated (and optimized) by, first, generating a predicted view based on the current framebuffer contents and correlating this prediction with the camera's captured, or observed, image. This may be accomplished by normalizing a correlation function, whereby an overall consistency measure is derived through randomly sampling points within a small mask of selected size of the predicted view and correlating them with corresponding points over a neighborhood in the observed/captured image. Poor correlation scores indicate that the projector has moved and re-calibration and geometric correction is required. Calibration consistency measures are sufficiently robust to distinguish small motion of the projector from continuously changing imagery.
As one will readily appreciate in connection with the instant technical disclosure, there are many fundamental distinguishing features of the instant invention from conventional methods. On occasion, the following manuscript authored by the applicants included herewith and labeled ATTACHMENT A (Steele and Jaynes, 2002) is referenced herein: Robert M. {Matt} Steele, and Jaynes, Christopher O., “Monitoring and Correction of Geometric Distortion in Projected Displays,” pgs. 1–8, Central European Conference on Computer Graphics and Computer Vision, 4–8 Feb. 2002; Steele and Jaynes, 2002 is incorporated by reference herein to the extent it provides technical background information, by illustrating the rigorous mathematical and engineering analyses performed by applicants of their unique technique and system.